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® Computer human interface. 



© In a computer human interface an adjustable "window " (177. FIG 4) enables the user to view a portion of an 
abstract, device-independent "picture" description of information. More than one window can be opened at a 
time. Each window can be sized independently of another, regardless of the applications running on them. The 
human interface creates a separate "object" (represented by a process) for each active picture and for each 
active window. The pictures are completely independent of each other. Multiple pictures (170. 174) can be 
updated simultaneously, and windows can be moved around on the screen and their sizes changed without the 
involvement of other windows and/or pictures. Images, including windows, representing portions of any or all of 
the applications can be displayed and updated on the output device simultaneously and independently of one 
another. All human interface with the operating system is performed through virtual input/output devices (186, 
187, FIG. 5), and the system can accept any form of real input or output devices. 
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COMPUTER HUMAN INTERFACE 



RELATED INVENTIONS 

The present invention is' related to the following inventions, all filed on May 6. 1985, and all assigned to 
5 the assignee of the present invention: 

1 . Title: Nested Contexts in a Virtual Single Machine 
Inventors: Andrew Kun, Frank Kolnick, Bruce Mansfield 
Serial No.: 730,903. 

2. Title: Computer System With Data Residence Transparency and Data Access Transparency 
io Inventors: Andrew Kun, Kolnick, Bruce Mansfield 

Serial No.: 730,929 

3. Title: Network Interface Module With Minimized Data Paths 
Inventors: Bernhard Weisshaar, Michael Barnea 

Serial No.: 730,621 

'5 4. Title: Method of Inter-Process Communication in a Distributed Data Processing System 

Inventors: Bernhard Weisshaar, Andrew Kun, Frank Kolnick, Bruce Mansfield 
Serial No.: 730,892 

5. Title: Logical Ring in a Virtual Single Machine 
Inventor: Andrew Kun, Frank Kolnick, Bruce Mansfield 

20 Serial No.: 730,923 

6. Title: Virtual Single Machine With Message-Like Hardware Interrupts and Processor Exceptions 
Inventors: Andrew Kun. Frank Kolnick, Bruce Mansfield 

Serial No.: 730.922 

The present invention is also related to the following inventions, all filed on even date herewith, and all 
25 assigned to the assignee of the present invention: 

7. Title: Self-Configuration of Nodes in a Distributed Message-Based Operating System 
Inventor: Gabor Simor 

Serial No.: 000,621 

8. Title: Process Traps in a Distributed Message-Based Operating System 
30 Inventors: Gabor Simor 

Serial No.: 000,624 



TECHNICAL FIELD 

35 

This invention relates generally to digital data processing, and. in particular, to a human interface 
system in which information is represented in at least one abstract, device-independent picture with a user- 
adjustable window onto such picture; to a human interface system in which images corresponding to 
multiple applications can be displayed and updated on a suitable output device simultaneously and 
40 independently of one another; to a human interface system providing means for converting "real" input into 
virtual input, and means for converting virtual output into "real" output; and to human interface system in 
which multiple applications are active in one or more independent pictures, can be updated simultaneously 
and independently of one another, and can be displayed in multiple independent "live" windows on a single 
screen. 

45 

BACKGROUND OF THE INVENTION 

It is known in the data processing arts to provide an output display device in which one or more 
so "windows" present information to the viewer. By means of such windows the user may view portions of 
several applications (e.g. word-processing, spreadsheet, etc.) simultaneously. However, in the known 
"windowing" art each window is necessarily of identical size. The ability to size each window independently 
to any desired dimension is at present unknown. 

There is therefore a significant need to be able to provide within the human interface of a data 
processing operating system the capability of adjusting the sizes of multiple windows independently of one 
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another. 

It is known in the data processing arts to provide an output display in which images from multiple 
applications can be displayed. For example, it is known to print a portion of a spread-sheet to disk and then 
read such portion into a desired place in a word-processing application file. In this manner, information from 
one application may be incorporated into another. 

However in the known technique for integrating information from two or applications, once the output of 
an application was printed to disk it was "dead" information and was no longer an active part of the 
application. Using the example given above, the spread-sheet portion would have been fixed in time and 
would no longer vary with a change in one of its cells. To reflect such a change, the spread-sheet would 
have had to be printed again to disk and then re-read into the word-processing file. 

There is therefore a significant need to be able to provide within the human interface of a data 
processing operating system the ability to permit information from multiple application sources to be 
displayed simultaneously in a live condition. 

It is further known in the data processing arts to couple a wide assortment of input and output devices 
to a data processing system for the purpose of providing an appropriate human interface. Such devices 
may take the form of keyboards of varying manufacture, "mice", touch-pads, joy-sticks, light pens, video 
screens, audio-visual signals, printers, etc. 

Due to the wide variety of I/O devices which can be utilized in the human/computer interface, it would 
be very desirable to isolate the human interface software from specific device types. The I/O should be 
independent of any particular "rear devices. 

There is thus a need for a computer human interface which performs I/O operations in an abstract 
sense, independent of particular "real" devices. 

It is also known in the data processing arts to provide an output display in which one or more 
"windows" present information to the viewer. By means of such windows the user may view portions of 
several applications (e.g. word-processing, spread-sheet, etc.) simultaneously. However in the known 
"windowing" art, only one window at a time may be "live" (i.e. responding to and displaying an active 
application). There is thus a significant need to be able to provide within the human interface of a data 
processing operating system the capability of displaying multiple "live" windows simultaneously. 

BRIEF SUMMARY OF INVENTION 

Accordingly, it is an object of the present invention to provide a data processing system having an 
improved human interface. 

It is further an object of the present invention to provide an improved data processing system human 
interface which allows a user to independently adjust the sizes of a plurality of windows appearing on an 
output device such as a video display unit or printer. 

It is also an object of the present invention to provide an improved human interface system which 
allows information from multiple applications to be integrated in a "live" condition on a single display. 

It is yet another object of the present invention to provide an improved human interface system which 
performs input/output operations in an abstract sense, independent of any particular I/O devices. It is 
another object of the present invention to provide an improved human interface system in which any type of 
"real" input and output devices may be employed, and which I/O devices may be connected to and 
disconnected from the data processing system without disrupting processing operations. 

It is additionally an object of the present invention to provide an improved human interface* system 
which allows the simultaneous display of separate "live" windows. 

It is another object of the present invention to provide a human interface system in which multiple 
applications represented by separate pictures may be active simultaneously. 

These and other objects are achieved in accordance with a preferred embodiment of the invention by 
providing a human interface in a data processing system, the interface comprising means for representing 
information in at least one abstract, device-independent picture, means for generating a first message, such 
first message comprising size information, and a console manager process responsive to the first message 
for creating a window onto the one picture, the size of the window being determined by the size information 
contained in the first message. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is pointed out with particularly in the appended claims. However, other features of the 
invention will become more apparent and the invention will be best understood by referring to the following 
5 detailed description in conjunction with the accompanying drawings in which: 

FIG. 1 shows a representational illustration of a single network, distributed message-based data 
processing system of the type incorporating the present invention. 

FIG. 2 shows a block diagram illustrating a multiple-network, distributed message-based data 
processing system of the type incorporating the present invention. 
*o 3 shows a standard message format used in the distributed data processing system of the 

present information. 

FIG. 4 shows the relationship between pictures, views, and windows in the human interface of a data 
processing system of the type incorporating the present invention. 

FIG. 5 shows a conceptual view of the different levels of human interface within a data processing 
75 system incorporating the present invention. 

FIG. 6 illustrates the relationship between the basic human interface components in a typical working 
environment. 

FIG. 7 shows the general structure of a complete picture element. 

FIG. 8 shows the components of a typical screen as contained within the human interface system of 
20 the present invention. 

FIG. 9 shows the relationship between pictures, windows, the console manager, and a virtual output 
manager through which multiple applications can share a single video display device, in accordance with a 
preferred embodiment of the present invention. 

FIG. 10 shows a flowchart illustrating how an application program interacts with the console manager 
25 process to create/destroy windows and pictures, in accordance with a preferred embodiment of the present 
invention. 

FIG. 1 1 illustrates an operation to update a picture and see the results in a window of selected size, 
in accordance with a preferred embodiment of the present invention. 

FIG. 12 illustrates how a single picture can share multiple application software programs. 
30 FIG. 13 illustrates how the picture manager multiplexers several applications to a single picture. 

FIG. 14 shows the live integration of two applications on a single screen within the human interface 
system of the present invention. 

FIG. 15 shows how the console manager operates upon virtual input to generate virtual output. 
FIG. 16 shows how virtual input is handled by the console manager. 
35 FIG. 17 shows how virtual input is handled by the picture manager. 

FIG. 18 illustrates flow the console manager enables multiple application, software programs to be 
represented by multiple pictures, and how multiple windows may provide different views of one picture. 
FIG. 19 illustrates how several windows may be displayed simultaneously on typical screen. 

40 

OVERVIEW OF COMPUTER SYSTEM 

The present invention can be implemented either in a single CPU data processing system or in a 
distributed data processing system - that is, two or more data processing system (each having at least one 

45 processor) which are capable of functioning independently but which are so coupled as to send and receive 
messages to and from one another. 

A Local Area Network (LAN) is an example of a distributed data processing system. A typical LAN 
comprises a number of autonomous data processing "nodes", each comprising at least processor and 
memory. Each node is capable of conducting data processing operations independently. 

so With reference to FIG. 1, a distributed computer configuration is shown comprising multiple nodes 2-7 
(nodes) loosely coupled by a local area network (LAN) 1. The number of nodes which may be connected to 
the network is arbitrary and depends upon the user application. Each node comprises at least a processor 
and memory, as will be discussed in greater detail with reference to FIG. 2 below. In addition, each node 
may also include other units, such as a printer 8, operator display module (ODM) 9, mass memory module 

55 13. and other I/O device 10. 

With reference now to Fig. 2. a multiple-network, distributed computer configuration is shown. A first 
focal area network LAN 1 comprises several nodes 2.4, and 7, LAN 1 is coupled to a second local area 
network LAN 2 by means of an Intelligent Communication Module (ICM) 50. The Intelligent-Communications 
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Module provides a link between the LAN and other networks and/or remote processors (such as program- 
mable controllers). 

LAN 2 may comprise several nodes (not shown) and may operate under the same LAN protocol as that 
of the present invention, or it may operate under any of several commercially available protocols, such as 
5 Ethernet; MAP, the Manufacturing Automatic Protocol of General Motors Corp.; Systems Network Architec- 
ture (SNA) of International Business Machines, Inc.; SECS-II; etc. Each ICM 50 is programmable for 
carrying out one of the above-mentioned specific protocols. In addition, the basic processing module of the 
node itseif can be used as an intelligent peripheral controller (IPC) for specialized devices. 

LAN 1 is additionally coupled to a third local area network LAN 3 via ICM 52. A process controller 55 is 
to also coupled to LAN 1 via ICM 54. 

A representative node N (7, FIG. 2) comprises a processor 24 which, in a preferred embodiment, is a 
processor from the Motorola 68000 family of processors. Each node further includes a read only memory 
(ROM) 28 and a random access memory (RAM) 26. In addition, each node includes a Network Interface 
Module (NIM) 21, which connects the node to the LAN, and a Bus Interface 29, which couples the node to 
ts additional devices within a node. While a minimal node is capable of supporting two peripheral devices, 
such as an Operator Display Module (OOM) 41 and an I/O Module 44, additional devices (including 
additional processors, such as processor 27) can be provided within a node. Other additional devices may 
comprise, for example, a printer 42, and a mass-storage module 43 which supports a hard disc and a back- 
up device (floppy disk or streaming tape drive). 
20 The Operator Display Module 41 provides a keyboard and screen to enable and operator to input 
information and receive visual information. 

The system is particularly designed to provide an integrated solution for office or factory automation, 
data acquisition, and other real-time applications. As such, it includes a full complement of service, such as 
a graphical output, windows, menus, icons, dynamic displays, electronic mail, event recording, and file 
25 management. 



SOFTWARE MODEL 

30 The computer operating system of the present invention operates upon processes, messages, and 
contexts, as such terms are defined herein. Thus this operating system offers the programmer a hardware 
abstraction, rather than a data or control abstraction. 

A "process", as used within the present invention, is defined as a self-contained package of data and 
executable procedures which operate on that data, comparable to a "task" in other known system. Within 

35 the present invention a process can be thought of as comparable to a subroutine in terms of size, 
complexity, and the way it is used. The difference between processes and subroutines is that processes 
can be created and destroyed dynamically and can execute concurrently with their creator and other 
"subroutines". 

Within a process, as used in the present invention, the data is totally private and cannot be accessed 
from the outside, i.e., by other processes. Processes can therefore be used to implement "objects ", 
"modules", or other higher-level data abstractions. Each process executes sequentially. Concurrency is 
achieved through multiple processes, possibly executing on multiple processes. 

Every process in the distributed data processing system of the present invention has a unique identifier 
(PID) by which it can be referenced. The P!0 is assigned by the system when the process is created, and it 
45 is used by the system to physically locate the process. 

Every process also has a non-unique, symbolic "name", which is a variable-length string of characters. 
In general, the name of a process is known system-wide. To restrict the scope of names, the present 
invention utilizes the concept of a "context". 

A "context" is simply a collection of related processes whose names are not known outside of the 
so context. Contexts partition the name space into smaller, more manageable subsystems. They also "hide" 
names, ensuring that processes contained in them do not unintentionally conflict with those in other 
contexts. 

A process in one context cannot explicitly communicate with, and does not known about, processes 
inside other contexts. All interaction across context boundaries must be through a "context process", thus 
55 providing a degree of security. The context process often acts as a switchboard for incoming messages, 
rerouting them to the appropriate sub-processes in its context. 

A context process behaves like any other process and additionally has the property that any processes 
which it creates are known only to itself and to each other. Creation of the process constitutes definition of a 

5 
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new context with the same as the process. 

A "message" is a buffer containing data which teils a process what to do and/or supplies it with 
information it needs to carry out its operation. Each message buffer can have a different length (up to 64 
kilobytes). By convention, the first field in the message buffer defines the type of message (e.g.. "read", 
5 "print", "status", "event", etc.). 

Messages are queued from one process to another by name of PID. Queuing avoids potential 
synchronization problems and is used instead of semaphores, monitors, etc. The sender of a message is 
free to continue after the message is sent When the receiver attempts to get a message, it will be 
suspended until one arrives if none are already waiting in its queue. Optionally, the sender can specify that 
to it wants to wait for a reply and is suspended until that specific message arrives. Messages from any other 
source are not dequeued until after that happens. 

Within the present invention, messages are the only way for two processes to exchange data. 
A "message" is a variable-length buffer (limited only by the processor's physical memory size) which 
carries information between processors. A header, inaccessible to the programmer, contains the destination 
75 name and the sender's PID. By convention, the first field in a message is a null-terminated string which 
defines the type of message (e.g.. "read", "status", etc.) Messages are" queued to the receiving process 
when they are sent. Queuing ensures serial access and is used in preference to semaphores, monitors, etc. 

Messages provide the mechanism by which hardware transparency is achieved. A process located 
anywhere in the system may send a message to any other process anywhere else in the system (even on 
20 another processor) if it knows the process name. This means that processes can be dynamically distributed 
across the system at any time to gain optimal throughput without changing the processes which reference 
them. Resolution of destinations is done by searching the process name space. 

25 OPERATING SYSTEM 

The operating system of the present invention consists of a kernel, plus a set of processes which 
provide process creation and termination, time management (set time, set alarm, etc.) and which perform 
node start-up and configuration. Drivers for devices are also implemented as processes (EESP's), as 
30 described above. This allows both system services and device drivers to be added or replaced easily. The 
operating system also supports swapping and paging, although both are invisible to applications software. 

Unlike known distributed computer systems, that of the present invention does not use a distinct "name 
server" process to resolve names. Name searching is confined to the kernel, which has the advantage of 
being much faster. 

as In general, there exists a template file describing the initial software and hardware for each node in the 
system. The template defines a set of initial processors (usually one per service) which are scheduled 
immediately after the node start-up. These processes then start up their respective subsystems. A node 
configuration service on each node sends configuration messages to each subsystem when it is being 
initialized, informing it of the devices it owns. Thereafter, similar messages are sent whenever a new device 
40 is added to the node or a device fails or is removed from the node. 

Thus there is no well-defined meaning for "system up" or "system down" - as long as any node is 
active, the system as a wholly may be considered to be "up". Nodes can be shut down or started up 
dynamically without affecting other nodes on the network. The same principle applies, in a limited sense, to 
peripherals. Devices which can identify themselves with regard to type, model number, etc. can be added 
45 or removed without operator intervention. 

FIG. 3 shows the standard format of a message in a distributed data processing system of the type 
incorporating the present invention. The message format comprises a message i.d. portion 150; one or 
more "triples" 151 , 153. and 155; and an end-of-message portion 160. Each "triple" comprises a group of 
three fields, such as fields 156-158. The first field 156 of "triple" 151, designated the PCRT field, 
so represents the name of the process to be created. The second field 157 of "triple" 151 gives the size of the 
data field. The third field 158 is the data field. 

The first field 159 of "triple" 153, designated the PNTF field, represents the name of the process to 
notify when the process specified in the PCRT field has been created. 

A message can have any number of "triples", and there can be multiple "triples" in the same message 
55 containing PCRT and PNTF fields, since several processes may have to be created (i.e. forming a context, 
as described hereinabove) for the same resource. 

As presently implemented, portion 150 is 16 bytes in length, field 156 is 4 bytes, field 157 is 4 bytes, 
field 158 is variable in length, and EOM portion 160 is 4 bytes. 

6 



BN80OCID: <£P_JQ274O07A4JL?> 



0 274 087 



HUMAN INTERFACE • GENERAL 

The Human Interface of the present invention provides a set of tools with which an end user can 
construct a package specific to his applications requirements. Such a package is referred to as a 

5 "metaphor", since it reflects the user's particular view of the system. Multiple metaphors can be supported 
concurrently. One representative metaphor is, for example, a software development environment. 

The purpose of the Human Interface is to allow consistent, integrated access to the data and functions 
available in the system. Since users' perceptions of the system are based largely on the way they interact 
with it, it is important to provide an interface with which they feel comfortable. The Human Interface allows a 

to system designer to create a model consisting of objects that are familiar to the end user and a set of 
actions that can be applied to them. 

The fundamental concept of the Human Interface is that of the "picture". All visually-oriented informa- 
tion, regardless of interpretation, is represented by pictures. A picture (such as a diagram, report, menu, 
icon, etc.) is defined in a device-independent format which is recognized and manipulated by all programs 

rs in the Human Interface and all programs using the Human Interface. It consists of "picture elements", such 
as "line", "arc", and "text", which can be stored compactly and transferred efficiently between processes. 
All elements have common attributes like color and fill pattern. Most also have type-specific attributes, such 
as typeface and style for text. Pictures are drawn in a large "world" co-ordinate system, composed of 
"virtual pixels". 

20 Because all data is in the form of pictures, segments of data can be freely copied between applications, 
e.g., from a live display to a word processor. No intermediate format or conversion is required. One 
consequence of this is that the end user or original equipment manufacturer (OEM) has complete flexibility 
in defining the formats of windows, menus, icons, error messages, help pages, etc. All such pictures are 
stored in a library rather than being built into the software and so are changeable at any time without 

25 reprogramming. A comprehensive editor is available to define and modify pictures on-line. 

All interaction with the user's environment is through either "virtual input" or "virtual output" devices. A 
virtual input device accepts keyboards, mice, light pens, analog diais, pushbuttons, etc. and translates them 
into text, cursor-positioning, action, dial, switch, and number messages. All physical input devices must map 
into this set of standard messages. Only one process, an input manager for the specific device, is 

30 responsible for performing the translation. Other processes can then deal with the input without being 
dependent on its source. 

Similarly, a virtual output manager translates standard output messages to the physical representation 
appropriate to a specific device (screen, printer, plotter, etc) A picture drawn on any terminal or by a 
process can be displayed or printed on any device, subject to the physical limitations of that device. 

35 With reference to FIG 4, two "pictures" are illustrated picture A (170) and picture 8 (174). 

The concept of a "view" is used to map a particular rectangular area of a picture to a particular device. 
In FIG. 4. picture A is illustrated as containing at least one view 171, and picture 8 contains at least one 
view 175. Views can be used, for example, to partition a screen for multiple applications or to extract page- 
sized subsets of a picture for printing. 

40 If the view appears on a screen it is contained in a "window. With reference again to FIG. 4, view 171 
of picture A is mapped to screen 1 76 as window 177, and view 175 of picture B is mapped as window 178. 

The Human Interface allows the user to dynamically change the size of the window, move the window 
around on the screen, and move the picture under the window to view different parts of it (i.e., scroll in any 
direction). If a picture which is mapped to one or more windows changes, all affected views of that pictures 

45 on all screens are automatically updated. There is no logical limit to the number or sizes of windows on a 
particular screen. Since the system is distributed, it's natural for pictures and windows to be on different 
nodes. For example, several alarm displays can share a single, common picture. 

The primary mechanism for interacting with the Human Interface is to move the cursor to the desired 
object and "select" it by pressing a key or button. An action may be performed automatically upon 

so selection or by further interaction, often using menus. For example, selecting an icon usually activates the 
corresponding application immediately. Selecting a piece of text is often followed by selection of a 
command such as "cut" or "underline". Actions can be dynamically mapped to function keys on a 
keyboard so that pressing a key is equivalent to selecting an icon or a menu item. A given set of cursors 
(the cursor changes as it moves from one application picture to another), windows, menus, icons, and 

55 function keys define a "metaphor". 

FIG. 5 shows the different levels of the Human Interface and data flow through them. Arrows 201-209 
indicate the most common paths, while arrows 210-213 indicate additional paths. The interface can be 
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configured to leave out unneeded layers for customized applications. The philosophy behind the Human 
Interface design dictates one process per object. That is, a process is created for each active window, 
picture, input or output device, etc. As a result, the processes are simplified and can be distributed across 
nodes almost arbitrarily. 

5 

MULTIPLE INDEPENDENT PICTURES AND WINDOWS 

A picture is not associated with any particular device, and it is of virtually unlimited size. A "window" is 
/o used to extract a specified rectangular area - called a "view" - of picture information from a picture and 
pass this data to a virtual output manager. 

The pictures are completely independent of each other. That is none is aware of the existence of any 
other, and any picture can be updated without reference to, and without affect upon, any other. The same is 
true of windows. 

/5 Thus the visual entity seen on the screen is really represented by two objects: a window (distinguished 
by its frame title, scroll bars, etc.), and a picture, which is (partially) visible within the boundaries of the 
window's frame. 

As a consequence of this autonomy, multiple pictures can be updated simultaneously, and windows can 
be moved around on the screen and their sizes changed without the involvement of other windows and/or 
20 pictures. 

Also, such operations are done without the involvement of the application which is updating the window. 
For example, if the size of a window is increased to look at a larger area of the picture, this is handled 
completely within the human interface. 

25 

HUMAN INTERFACE - PRIMARY FEATURES 

The purpose of the Human Interface is to transform machine-readable data into human-readable data 
and vice versa. In so doing the Human Interface provides a number of key services which have been 
30 integrated to allow users to interact with the system in a natural and consistent manner. These features will 
now be discussed. 

Device Independence -The Human Interface treats all devices (screens printers, etc.) as "virtual 
devices". None of the text, graphics, etc. in the system are tied to any particular hardware configuration. As 
a result such representative can be entered from any "input" device and displayed on any "output" device 

35 without modification. The details of particular hardware idiosyncrasies are hidden in low-level device 
managers, all of which have the same interface to the Human Interface software. 

Picture Drawing -The Human Interface can draw "pictures" composed of any number of geometric 
elements, such as lines, circles, rectangles, etc., as well as any arbitrary shape defined by the user. A 
picture can be of almost any size. All output from the Human Interface to the user is via pictures, and all 

40 input from a user to the Human Interface is stored as pictures, so that there is only one representation of 
data within the Human Interface. 

Windowing -The Human Interface allows the user to partition a screen into as many "sub-screens" or 
"windows" as required to view the information he desires. The Human Interface places no restrictions on 
the contents of such windows, and all windows can be simultaneously updated in real time with data from 

45 any number of concurrently executing programs. Any picture can be displayed, created, or modified 
("edited") in any window. Also any window can be expanded or contracted, or it can be moved to a new 
location on the screen at any time. 

If the current picture is larger than the current window, the window can be scrolled over the picture, 
usually in increments of a "line" or a "page". It is also possible to temporarily expand or contract the visible 

so portion of the picture ("zoom in" or "zoom out") without changing the window's dimensions and without 
changing the actual picture. 

Dialog Management -The Human Interface is independent of any particular language or visual repre- 
sentation. That is. there are not built-in titles, menus, error messages, help text, icons, etc. for interacting 
with the system. All such information is stored as pictures which can be modified to suit the end user's 

55 requirements either prior to or after installation. The user can modify the supplied dialog with his own at any 
time. 

Data Entry -The Human Interface provides a generalized interface between the user and any program 
(such as a data base manager) which requires data from the user. The service is called "forms 

8 
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management", because a g IV en data structure is displayed as a fill-in-the-blanks type of "form" consisting 
of numerous modifiable fields with description labels. The Human Interface form is interactive, so that data 
can be verified as it is entered, and the system can assist the user by displaying explanatory test when 
appropriate (on demand or as a result of an error). 



HUMAN INTERFACE - BASIC COMPONENTS 

The Human Interface comprises the following basic components: 
w Console Manager -It is the central component of a Console context and consequently is the only 
manager which knows all about its particular "console". It is therefore aware of all screens and keyboards, 
all windows, and all pictures. Its primary responsibility is to coordinate the activities of the context. This 
4 consists of starting up the console (initializing the device managers, etc.) creating and destroying pictures, 

and allocating and controlling windows for processes in the Human Interface and elsewhere. Thus all access 
/s to a console must be indirect, through the relevant Console Manager. 

Console Manager also implements the first level of Human Interface interaction, via menus, prompts, 
etc., so that applications processes don't have to. Rather than using built-in text and icons, it depends upon 
the Dialog Manager to provide it with the visible features of the system. Thus all cultural and user 
idiosyncrasies (such as language) are hidden from the rest of the Human Interface. 
20 A Console Manager knows about the following processes: the Output Manager(s) in its context, the 
Input Manager in its context, the Window Manager in its context, the Picture Managers in its context, and 
the Dialog Manager in its context. The following processes know about the Console Manager: any one that 
wants to. 

When a Console Manager is started, it waits for the basic processes needed to communicate with the 
25 user to start up and "sign on". It this is successful it is ready to talk to users and other processes (i.e., 
accept messages from the Input Manager and other processes). AH* other permanent processes in the 
context (Dialog, etc.) are assumed to be activated by the system start-up procedure. The "IN" and "Cursor" 
processes (see "Input Manager" and "Output Manager" below) are created by the Console Manager at this 
time. 

30 The Console Manager views the screen as being composed of blank (unused) space, windows, and 
icons. Whenever an input character is received, the Console Manager determines how to handle it 
depending upon the location of the cursor and the type of input, as follows: 

A. Requests to create or eliminate a window are handled within the Console Manager. A window may 
be opened anywhere on the screen, even on top of another window. A new Picture Manager and possibly a 
35 Window Manager may be created as a result, and one or more new messages may be generated and sent 
to them, or the manager(s) may be told to quit. 

8. Icons can only be selected, then moved or opened. The Console Manager handles selection and 
movement directly. It sends notification of an "open" to the Dialog Manager, which sends a notification to 
the application process associated with the icon and possibly opens a default window for it. 
<*o C. For window-dependent actions, if the cursor is outside all windows, the input is illegal, and the 

Console Manager informs the user; otherwise the input is accepted. Request which affect the window itself 
(such as "scroll" or "200m") are handled directly by the Console Manager. A "select" request is pre- 
checked. the relevant picture elements are selected (by sending a message to the relevant Picture 
Manager), and the passage is passed on to the process currently responsible for the windows. All other 
45 inputs are passed directly to the responsible process without being pre-checked. 

If the cursor is on a window's frame, the only valid actions are to move, close, or change the 
dimensions of the window, or select an object in the frame (such as a menu or a scroll bar). These are 
handled directly by the Console Manager. 

A new window is opened by creating a new Window Manager process and telling it its dimensions and 
so the location of its upper left corner on the screen. It must also be given the PID of a Picture Manager and 
the coordinates of the part of the picture it is to display, along with the dimensions of a "clipping polygon", 
if that information is available (It is not possible to create a window without a picture). The type and contents 
of the window frame are also specified. Any of these parameters may be changed at any time. 

A new instance of a picture is created by creating a new Picture Manager process with the appropriate 
55 name and. optionally, telling it the name of a "file" from which to get its picture elements. If a file is not 
provided, an "empty" picture is created, with the expectation that picture-drawing requests will fill it in. . 

Menus, prompts, help messages, error text, and icons are simply predefined pictures (provided through 
the Dialog Manager) which the Console Manager uses to interact with users. They can therefore be created 

9 



BN80OCID: «£P__JH740e7A2JL> 



0 274 087 



and edited to meet the requirements of any particular system the same way any picture can be created and 
edited. Menus and help text and usually displayed on request, although they may sometimes be a result of 
another operation. 

Picture Manager -It is created when a picture is built, and it exits when the picture is no longer 
5 required. There is one Picture Manager per picture. The Picture Manager constructs a device-independent 
representation of a picture using a small set of elemental "picture elements" and controls modification and 
retrieval of the elements. 

A Picture Manager knows about the following processes: the process which created it, and the Draw 
Manager. The fofiowing processes know about the Picture Manager: the Console Manager in the same 
to context, and Window Managers in the same context. 

A Picture Manager is created to handle exactly one picture, and it need only be carried when the 
picture is being accessed. It can be told to quit at any time, deleting its representation of the picture. Some 
other process must copy the picture to a fife if it needs to be saved. 

When a Picture Manager first starts up, its internal picture is empty. It must receive a "load file" 
ts request, or a series of "draw" requests, before a picture is actually available. Until that is done any requests 
which refer to specific elements or locations in the picture will receive an appropriate "not found" status 
message. 

A picture is logically composed of device-independent "elements", such as text, line, arc, and symbol. 
In general, there is a small number of such elements. Each element consists of a common header, which 
20 includes the element's position in the picture's coordinate system, its color, size, etc.. and a "value" which 
is unique to the element's type (e.g. a character string, etc.). The header also specifies how the element 
combines with other elements in the picture (overlays them, merges with them, etc.). 

Input Manager -There is one Input Manager per set of "logical input devices" (such as keyboards, 
mice, light, pens, etc.) connected to the system. The Input Manager handles input interrupts and passes 
25 them to the console manager. Cursor movement inputs may also be sent to a designated output manager. 

The Input Manager knows about the following processes: the process which initialized it, and possibly 
one particular Output Manager in the same context. The following process knows about the Input Manager: 
the Console Manager in the same context. 

An input Manager is created (automatically, at system start-up) for each set of "logical input devices" in 
go the system, thus implementing a single "virtual keyboard". There can only be one such set, and therefore 
one Input Manager, per Console context. The software (message) interface to each manager is identical, 
although their internal behaviour is dependent upon the physical device(s) to which they communicate. All 
input devices interrupt service routines (including mouse, digitizing pad, etc.) are contained in Input 
Manager and hidden from other processes. When ready, each Input Manager must send an "I'm here" 
as message to the closest process named "Console". 

An Input Manager must be explicitly initialized and told to proceed before it can begin to process input 
interrupts. Both of these are performed using appropriate messages. Whichever, process initializes the 
manager becomes tightly coupled to it, i.e., they can exchange messages via PfD's rather than by name. 
The Input Manager will send all inputs to this process (usually the Console Manager). This coupling cannot 
40 be changed dynamically; the manager would have to be re-initialized. Between the "initialize" and the 
"proceed" an Input Manager may be sent one or more "set" requests to define its behaviour. It does not 
need to be able to interpret the meaning of any input beyond distinguishing cursor for non-cursor. Device- 
independent parameters (such as pixel size and density) and not down-loaded but rather are assumed to be 
built into the software, some part of which, in general, must be unique to each type of Input Manager. 
45 An Input Manager can be dynamically "linked" to a particular Output Manager, if desired. If so. all 

cursor control input (or any other given subset of the character set) will be sent to that manager, in addition « 
to the initializing process, as it is received. This assignment can be changed or cut off at any time. (This is 
generally useful only if the output device is a screen. 

In general, input is sent as signal "characters", each in a single "K" (i.e. keyboard string) message * 
50 (unbuffered) to the specified process(es). Some characters, such as "shift one" or a non-spacing accent, 
are temporarily buffered until the next character is typed and are then sent as a pair. Redefinable 
characters, including all displayable text, cursor control commands, "action keys", etc. are sent as triples. 

New outputs devices can be added to the "virtual keyboard" at any time by re-initializing the manager 
and down-loading the appropriate parameters, followed by a "proceed". All input is suspended while this is 
55 being done. Previously down-loaded parameters and the screen assignment are not affected. Similarly, 
devices can be disconnected by terminating (sending "quit" requests for) them individually. A non-specific 
"quit" terminates the entire manager. 

Where applicable, an Input Manager will support requests to activate outputs on its device(s). such as 
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lights or sound generators (e.g., a bell). 

The input Process is a distinct process which is created by each Console Manager for its Input 
Manager to keep track of the current input state, in general, this includes a copy of its last input of each 
type (text, function key, pointer, number, etc.), the current redefinable character set number, as well as 
5 Boolean variables for such conditions as "keyboard locked", "select key depressed" (and being held 
down), etc. The process is simply named "In". The Input Manager is responsible for keeping this process 
up-to-date. Any process may examine (but not modify) the contents of "In". 

Output Manager -There is one Output Manager per physical output device (screen, printer, plotter, 
etc.) connected to the system. Each Output Manager converts (and possibly scales) standard "pictures" 
to into the appropriate representation on its particular device. 

The Output Manager knows about the following processes: the process which initialized it, and the Draw 
Manager in the same context. The following processes know about the Output Manager: the Console 
Manager in the same context, the Input Manager in the same context, and the Window Manager in the 
same context. 

is An Output Manager is created (automatically, at system start-up) for each physical output device in the 
system, thus implementing numerous "virtual screens". There can be any number of such devices per 
Console context The software (message) interface to each manager is identical, although their internal 
behavior is dependent upon the physical device(s) to which they communicate. AH output interrupt service 
routines (if any) are contained in Output Manager and hidden from other processes. Each manager also 

20 controls a process called Cursor which holds information concerning its own cursor. When ready, each 
Output Manager must send an "I'm here" message to the closest process named "Console". 

An Output Manager must be explicitly initialized and told to proceed before it can begin to actually write 
to its device. Both of these are performed using appropriate Human interface messages. Which process 
initializes the manager becomes tightly coupled to it: i.e.. they can exchange messages via PID's rather 

25 than by name. This coupling cannot be changed dynamically: the manager would have to be re-initialized. 
Between the "initialize" and the "proceed" an Output Manager may be sent one or more "Set" requests to 
define its behaviour. Device-independent parameters (such as pixel size and density) are not down-loaded 
but rather are assumed to be built into the software, some part of which, in general, must be unique to each 
type of Output Manager. Things like a screen's background color and pattern are down-loadable at start-up 

30 time and at any other time. 

in general, an Output Manager is driven by "draw" commands (containing standard picture elements) 
sent to it by any process (usually a Window Manager). Its primary function then is to translate picture 
elements, described in terms of virtual pixels, into the appropriate sequences of output to its particular 
device. It uses the Draw Manager to expand elements into sets of real pixels and keeps the Cursor process 

35 informed of any resulting changes in cursor position. It looks up colors and shading patterns in predefined 
tables. The "null" color (zero) is interpreted as "draw nothing" whenever it is encountered. A "clear" 
request is also supported. It changes a given polygonal area to the screen's default color and shading 
pattern. 

The Cursor Process is a distinct process which is created by each Console Manager in its context to 
<*o keep track of the cursor. That process, which has the same name as the screen (not the Output Manager), 
knows the current location of the cursor, all of the symbols which may represent the cursor on the screen, 
which symbol is currently being used, how many reai pixels to move when a cursor movement command is 
executed, etc. it can, in general, be accessed for any of this information at any time by any process. The 
associated Output Manager is the prime user of the process and is responsible for keeping it up to date. 
45 The associated input Manager (if any) is the next most common user, requesting the cursor's position every 
time it processes a "command" input 

Dialog Manager -There is one Dialog Manager per console, and it provides access to a library of 
"pictures" which define the menus, help texts, prompts, etc. for the Human Interface (and possibly the rest 
of the system), and it handles the user information with those pictures, 
so The Dialog Manager knows about the following processes: none. The following processes know about 
the Dialog Manager: the Console Manager in the same context. 

One Dialog Manager is created automatically, at system start-up, in each Console context. Its function 
is to handle all visual interaction with users through the input and output managers. Its purpose is to 
separate the external representation of such interaction from its intrinsic meaning. For example, the Console 
55 Manager may need to ask the user how many copies of a report he wants. The phasing of the question and 
. the response are irrelevant - they may be in English, Swahili, or pictographic, so long as the Console 
Manager ends up with an integral number of perhaps the response "forget it". 

in general, the Dialog Manager can be requested to load (from a file) or dynamically create (from a 

1 1 



BN80OCID: <£P QZ74067A2_JU> 



0 274 087 



given specification) a picture which represents a menu, error message, help (informational) text, prompt, a 
set of icons, etc. This picture is usually displayed until the user responds. 

Response to help or error text is simply acknowledgement that the text has been read. The response to 
a prompt is the requested information. The user can respond to a menu by selecting an item in the menu or 
5 by canceling the menu (and thus canceling any actions the menu would have caused). Icons can be 
selected and then moved or "opened". Opening an icon generally results in an associated application being 
run. 

"Selection" is done through an Input Manager which sends a notification to the Console Manager. The 
Console Manager filters this response through the Dialog Manager which interprets it and returns the 
to appropriate parameter in a message which is then passed on to the process which requested the service. 

All dialog is represented as pictures, mostly in free format. Help and error dialog are the simplest and 
are unstructured except that one element must be "tagged" to identify it as the "I have read this text" 
response target symbol. The text is displayed until the user selects this element. 

Draw Manager -There is one Draw Manager per console, and it provides access to a library of 
is "pictures" which define the menus, help, prompts, etc., for the Human Interface (and possibly the rest of 
the system), and it handles the user interaction with those pictures. 

The Draw Manager knows about the following processes: none. The following processes know about the 
■ Draw Manager: the Picture Mangers in the same context, and the Output Managers in the same context 

One Draw Manager is created automatically, at system start-up, in each context that requires expansion 
20 of picture elements into bit-maps. Its sole responsibility is to accept one or more picture elements, of any 
type, in one message and return a list of bit-map ("symbol") elements corresponding to the figure 
generated by the elements, also in one message. Various parameters can be applied to each element, most 
notably scaling factors which can be used to transform an element or to convert virtual pixels to real pixels. 
The manager must be told to exit when the context is being shut down. 
25 Window Manager -There is one per current instance of a "window" on a particular screen. A Window 
Manager is created when the window is opened and exits when the window is closed. It maps a given 
picture (or portion thereof) to a rectangular area of a given size on the given screen; i.e., it fogically links a 
device-independent picture to a device-dependent screen. A "frame" can be drawn around a window, 
marking its boundaries and containing other information, such as a title or menu. Each manager is also 
30 responsible for updating the screen whenever the contents of its window changes. 

The Window Manger knows about the following processes: the process that created it; one particular 
Picture Manager in the same context; and one particular Picture Manager in the same following processes 
know about the Window Manager: the Console Manager in the same context. 

The Window Manager's main job is to copy picture elements from a given rectangular area of a picture 
35 to a rectangular area (called a "window") on a particular screen. To do so it interacts with exactly one 
Picture Manager and one Output Manager. A Window Manager need only be created when a window is 
"opened" on the screen and can be told to quit when the window is "closed" (without affecting the 
associated picture). When opened, the Motorola must draw the outline, frame, and background of the 
window. When closed, the window and its frame must be erased (i.e. redrawn in the screen's background 
40 color and pattern). "Moving" a window (changing its location on the screen) is essentially the same as 
closing and re-opening it. 

A Window Manager can only be created and destroyed by a Console Manager, which is responsible for 
arranging windows on the screen, resolving overlaps, etc. When a Window Manger is created, it waits for an 
"initialize" message, initializes itself, returns an "I'm here" message to the process which sent it the 

45 "initialize" message, then waits for further messages. It does not send any messages to the Output 
Manager until it has received all of the following: its dimensions (exclusive of frame), the outline line-type, 
size and color, background color, location on the screen, a clipping polygon, scaling factors, and framing 
parameters. A Window Manager also has a "owner", which is a particular process which will handle 
commands (through the Console Manager, which always has prime control) within the window. 

so Any of the above parameters can be changed at any time. In general, changing any parameter (other 
than the owner) causes the window to be redrawn on the screen. 

A "frame", which may consist of four components (called "bars"), one along each edge of the window, 
may be placed around the given window. The bars are designated top, bottom, left, and right. They can be 
any combination of simple line segment, title bar, scroll bar, menu bar, and palette bar. These are supplied 

55 to the message as four separate lists (in four separate messages) of standard picture elements, which can 
be changed at any time by sending a new message referencing the bar. The origin of each bar is [0,0] 
relative to the upper left corner of the window. 

The Console Manager may query a Window Manager for any of its parameters, to which it responds 
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with messages identical to the ones it originally received. It can also be asked whether a given absolute 
cursor position is inside its window (i.e. inside the current clipping polygon) or its frame.-and for the cursor 
coordinates relative to the origin of the window or any edge of the frame. 

A Window Manager is tightly coupled to its creator (a Console Manager), Picture Manager, and Output 
s Manager, i.e. they communicate with each other using process identifiers (PID's). Consequently, a Window 
Manager must inform its Picture Manager when it exits, and it expects the Picture Manager to do the same. 

Once the Window Manger knows the picture it is accessing and the dimensions of its window (or any 
time either of these changes), it requests the Picture Manager to send to ail picture elements which 
completely or partially lie within the window. It also asks it to notify it of changes which will affect the 
jo displayed portion of the picture. The Picture Manager will send "draw" messages to the Window Manager 
(at any time) to satisfy these requests. 

The Window Manager performs gross clipping on all picture elements it receives, i.e. it just determines 
whether each element could appear inside the current clipping polygon (which may be smaller than the 
window at any given moment, if other windows overlap this one). 
'5 Window Managers deal strictly in virtual pixels and have no knowledge about the physical characteris- 
tics of the screen to which they are writing. Consequently, a window's size and location are specified in 
virtual pixels, implying a conversion from real pixels if these are different. 

Print Manager -There is one per rf Output subsystems", i.e. per pool of output devices. The Print 
Manager coordinates output to hard-copy devices (i.e. to their Output Managers). It provides a comprehen- 
20 sive queuing service for files that need to be printed, it can also perform some minimal formatting of text 
(justification, automatic page numbering, header, footers, etc.) 

The Print Manager knows about the following processes: Output Managers in the same context, and a 
Picture Manager in the same context. The following processes know about the Print Manager: any one that 
wants to. 

25 One Print Manager is created automatically, at start-up time, in each Print context. It is expected to 
accept general requests for hard-copy output and pass them on, one message (usually corresponding to 
one "line" or output) at a time, to the appropriate Output Manager. It can also accept requests which refer 
to files (i.e. to File Manager processes). Each such message, known as a "spool", request, also contains a 
priority, the number of copies desired, specific output device requirements (if any) and special form 

30 requirements (if any). Based on these parameters, as well as the size of the Hie, the amount of time the 
request has been waiting, and the availability of output devices, the Print Manager maintains an ordered 
queue of outstanding requests. It dequeues them one at a time, select an Output Manager, and builds a 
picture {using a Picture Manager). It then requests (from the Picture Manager) and "prints" (plots, etc) one 
"page" at a time until the entire file has been printed. 

35 

HUMAN INTERFACE - RELATIONSHIP BETWEEN COMPONENTS 

The eight Human Interface components together provide all of the services required to support a 
40 minimal human interface. The relationship between them are illustrated in FIG. 6, which shows at least one 
instance of each component. The components represented by circles 301, 302, 307, 312. 315. and 317-320 
are generally always present and active, while the other components are created as needed and exit when 
they have finished their specific functions. FIG. 6 is divided into two main contexts: "Console" 350 and 
"Print" 351. 

^5 Cursor 314 and Input 311 are examples of processes whose primary function is to store data. 
"Cursor's purpose is to keep track of the current cursor position on the screen and all parameters (such as 
the symbols defining different cursors) pertinent to the cursor. One cursor process is created by the 
Console Manager for each Output Manager when it is initialized. The Output Manager is responsible for 
updating the cursor data, although "Cursor" may be queried by anyone. "Input" keeps track of the current 

so input state, such as "select key is being held down", "keyboard locked", etc. One input process is created 
by each Console Manager. The console's input message updates the process; any other process may 
query it. 

The Human Interface is structured as a collection of subsystems, implemented as contexts, each of 
which is responsible for one broad area of the interface. There are two major contexts accessible from 
55 outside the Human Interface: "Console" and "Print". They handle all screen/keyboard interaction and all 
hard-copy output, respectively. These contexts are not necessarily unique. There may be one or more 
instances of each in the system, with possibly several on the same cell. Within each, there may be several 
levels of nested contexts. 
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The possible interaction between various Human Interface components will now be described. 

Console Manager/ Other Contents -Processes of other contexts may send requests for console 
services or notification of relevant events directly to the Console Managers). The Console Manager routes 
messages to the appropriate service. It also notifies (via a "status" message) the current owner of a window 
whenever an object in its window has been selected. Similarly, it sends a message to an application when a 
user requests that application in a particular window. 

Console Manager . Input Manager -The Console Manager initializes the Input Manager and usually 
assigns a particular Output Manager to it The Input Manager always sends all input (one character, one 
key. one cursor movement, etc. at a time) directly to the Console Manager. It may also send "status" 
messages, either in response to a "download", "initialize", or "terminate" request, or any time an anomaly 
arises. 

Console Manager ■ Output Manager -The Console Manager displays information on its "prime" output 
device during system start-up and shut-down without using pictures and windows. It therefore sends picture 
elements directly to an Output Manager. The Console Manager is also responsible for moving the cursor on 
the screen while the system is running, if applicable. The Console Manager (or an other Human Interface 
manager, such as an "editor") may change the current cursor to any displayable symbol. Output Managers 
will send "status" messages to the Console Manager any time an anomaly arises. 

Console Manager I Picture Manager -The console Manager creates Picture Managers on demand and 
tells each of them the name of a file which contains picture elements, if applicable. A Picture Manager can 
also accept requests from the Console Manager (or anyone else) io add elements to a picture individually, 
delete elements, copy ihem, move them, modify their attributes, or transform them, It can be queried for the 
value of an element at (or close to) a given location within its picture. The Console Manager will tell a 
Picture Manager to erase its picture and exit when it is no longer needed. A Picture Manager usually sends 
"Status" messages to the Console Manager whenever anything unusual (e.g., an error) occurs. 

Console Manager ■ Window Manager -The Console Manager creates Window Managers on demand. 
Each Window Manager is told its size, the PID of an Output Manager, the coordinates (on the screen) of its 
upper left outside corner, the characteristics of its frame, the PID of a particular Picture Manager, the 
coordinates of the first element from which to start displaying the picture, and the name of the process 
which "owns" the window. While a window is active, it can be requested to re-display the same picture 
starting at a different element or to display a completely different picture. 

The coordinates of the window itself may be changed, causing it to move on the screen, or it may be 
told to change it size, frame, or owner. A Window Manager can be told to "clip" the picture elements in its 
display along the edge of a given polygon (the default polygon is the inside edge of the window's frame). It 
can also be queried for the element corresponding to a given coordinate. The Console Manager will tell a 
Window Manger to "close" (erase) its window and exit when it is no longer needed. A Window Manager 
sends "status" messages to the Console Manager to indicate success or failure of a request. 

Console Manager > Dialog Manager -The Dialog Manager accepts requests to load and/or dynamically 
create "pictures" which represent menus, prompts error messages, etc. In the case of interactive pictures 
(such as menus), it also interprets the response for the Console Manager. Other processes may also use 
the Dialog Manager through the Console Manager. 

Console Manager / Print Manager -Console Managers generally send "spool" requests to Print 
Managers to get hard-copies of screens or pictures. An active picture must first be copied to a file. The 
Print Manager returns a "status" message when the request is complete or if it fails. 

Window Manager t Picture Manager -A Window Manager requests lists of one or more picture elements 
from the relevant picture Manager, specified by the coordinates of a rectangular "viewport" in the picture. It 
can also request the Picture Manager to automatically send changes (new. modified, or erased elements), 
or just notification of changes, to it. The Picture Manager sends "status" messages to notify the Window 
Manager of changes or errors. 

Window Manager Output Manager -A Window Manager sends lists of picture elements to its Output 
Manager, prefixed by the coordinates of a polygon by which the Output Manager is to "clip" the pixels of 
the elements as it draws them. A given list of picture elements can also be scaled by a given factor in any 
of its dimensions. The Output Manager returns a "status" message when a request fails. 

Input Managerl Output Manager -The Input Manager sends all cursor movement inputs to a pre- 
assigned Output Manager (if any), as well as to the Console Manager. This assignment can be changed 
dynamically. 

Print Manager • Other Processes -The Print Manager accepts requests to "spool" a file or to "print" 
one or more picture elements. It sends a "status" message at the completion of the request or if the 
request cannot be carried out. The status of a queued request can also be queried or changed at any time. 
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Print Manager File Manager -The Print Manager reads picture elements from a File Manager (whose 
name was sent to it via a "spoor request). It may send a request to "delete" the file back to the File 
Manager after it has finished printing the picture. 

Print Manager ■ Picture Manager -A Print Manager creates a Picture Manager for each spooled picture 
5 that it is currently printing, giving it the name of the relevant fiie. It then requests "pages'* of the picture 
(depending upon the characteristics of the output device) one at a time. Finally, it tells the Picture Manager 
to go away. 

Print Manager Output Manager -The Print Manager sends picture elements to an Output Manager. 
The Output Manager sends a "status" message when the request completes, or faiis or when an anomaly 
?o arises on the printer. 

Draw Manager : Other Processes -The Draw Manager accepts lists of elements prefixed by explicit 
pixel parameters (density, scaling factor, etc.). It returns a single message containing a list of bit-map 
("symbol") elements of the draw result for each message it receives. 

HUMAN INTERFACE - SERVICE 

A Human Interface service is accessed by sending a request message to the closest (i.e. the "next") 
Human Interface manager, or directly to a specific Console Manager. This establishes a "connection" on an 
20 existing Human Interface resource or creates a new one. Subsequent requests must be made directly to the' 
resource, using the connector returning from the initial request, until the connection is broken. The Human 
Interface manager is distributed and thus spans the entire virtual machine. Resources are associated with 
specific nodes. 

A picture may be any size, often larger than any physical screen or window. A window may only be as 
25 large as the screen on which it appears. There may be any number of windows simultaneously displaying 
pictures on a single screen. Updating a picture which is mapped to a window causes the screen display to 
be updated automatically. Several windows may be mapped to the same picture concurrently - at different 
coordinates. 

The input model provided by the Human Interface consists of two levels of "virtual devices". The lower 
30 level supports "position", "character", "action", and "function key" devices associated with a particular 
window. These are supported consistently regardless of the actual devices connected to the system. 

An optional higher level consists of a "dialog service", which adds "icons", "menus", "prompts", 
"values", and "information boxes" to the repertoire of device-independent interaction. Input is usually event- 
driven (via messages) but may also be sampled or explicitly requested. 
35 AH dimensions are in terms of "virtual pixels". A virtual pixel is a unit of measurement which is 

symmetrical in both dimensions. It has no particular size. Its sole purpose is to define the spatial 
relationships between picture elements. Actual sizes are determined by the output device to which the 
picture is directed, if and when it is displayed. One virtual pixel may translate to any multiple, including 
fractions, of a real pixel. 

40 Using the core Human Interface service generally involves: creating a picture (or accessing a 
predefined picture); creating a window on a particular screen and connecting the picture to it; updating the 
picture (drawing new elements, moving or erasing old ones, etc.) to reflect changes in the application (e.g. 
new data); if the application is interactive, repeatedly accepting input from the window and acting 
accordingly; and deleting the picture and/or window when done. 

45 Creating a new resource is done with an appropriate "create" message, directed to the appropriate 
resource manager (i.e. the Human Interface manager or Console Manager). Numerous options are available 
when a resource, particularly a window, is created. For example, a typical application may want to be 
notified when a specific key is pressed. Pop-up and pull-down menus, and function keys, may also be 
defined for a window. 

so All input from the Human Interface is sent by means of the "click" message. The input of this message 
is to allow the application program to be as independent of the external input as possible. Consequently, a 
"click" generated by a pop-up menu looks very much like that generated by pressing a function key or 
selecting an icon. Event-driven input is initiated by a user interacting with an external device, such as a 
keyboard or mouse. In this case, the "click" is sent asynchronously, and multiple events are queued. 

55 A program may also explicitly request input, using a menu, prompt, etc., in which case the "click" is 
sent only when the request is satisfied. A third method of input, which doesn't directly involve the user, is to 
query the current state of a virtual input device (e.g., the current cursor position). 
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A "click" message is associated with a particular window (and by implication usually with a particular 
picture), or with a dialog "metaphor", thus reflecting the two levels of the input model. 

Since the visual aspect of the Human Interface is separated from the application aspect, a later 
redesign of a window, menu, icon, etc. has little or no effect upon existing applications. 

5 

HUMAN INTERFACE - DETAILED DESCRIPTION 
CONNECTORS 

jo 

In general, all interaction with a Human Interface resource (console, window, picture, or virtual terminal) 
must be through a connector to that resource. Connectors to consoles can only be obtained from the 
Human Interface manager. Connectors to the other resources are available through the Human Interface 
manager, or through the Console Manager in which the desired resource resides. Requests must specify 
is the path-name of the resource as follows: 

[<console name>] (/<screen name>] [;<window or__picture name>] 

That is, the name of the console, optionally followed by a slash and the name of the screen, optionally 
followed by a slash and the name of a window, picture, or terminal. The console name may be omitted only 
if the message is sent directly to the desired console manager. If the screen name is omitted, the first 
20 screen configured on the given console is assumed. The window name must be specified if one of those 
resources is being connected. 

CONNECTION REQUESTS 

25 

The "create" and "open" requests can be addressed to the "next" Human Interface context ("HI") or to 
a specific console connector or to the "next" context named "Console". If sent to "HI", a full path-name 
(the name parameter) must be given; otherwise, only the name of the desired resource is required (e.g., at 
a minimum, just the name of the window or picture). 
30 If a picture manager process is created locally by an application, for private use, an "init" message - 
with the same contents as "create" or "open" - must be sent directly to the picture process. The response 
will be "done" or "failed". 

The following are the various Connection Requests and the types of information which may be 
associated with each: 

35 CREATE is used to create a new picture resource, a new window resource, or a new virtual terminal 
resource. 

When used to create a new picture resource, it may contain information about the resource type (i.e. a 
"picture"); the path-name of the picture; the size; the background color; the highlighting method; the 
maximum number of elements; the maximum element size; and the path-name of a library picture from 

40 which other elements may be copied. 

When used to create a new window resource, it may contain information about the resource type (i.e. a 
"window"); the path-name of the window; the window's title; the window's position on the screen; the size of 
the window; the color, width, fill color between the outline and the pane, and the style of the main window 
outline; the color and width of the pane outline; a mapping of part of a picture into the window; a 

45 modification notation: a special character notation; various options; a "when" parameter requesting notifica- 
tion of various specified actions on/within the window; a title bar; a palette bar; vertical and horizontal scroll 
bars; a general use bar; and a corner box. 

When used to create a new virtual terminal, it may contain information about the resource type (i.e. a 
"terminal"); the path-name of the terminal; the title of the terminal's window; various options; the terminal's 

so position on the screen; the size of the terminal (i.e. number of lines and columns in the window); the 
maximum height and width of the virtual screen; the color the text inside the window; tab information; 
emulator process information; connector information to an existing window; window frame color; a list of 
menu items: and alternative format information. 

OPEN is used to connect to a Human Interface service or to an existing Human Interface resource. 

ss When used to connect to a Human Interface service, it may contain information about the service type; and 
the name of the particular instance of the service. This resource must be sent to the Human Interface 
context. 

When used to connect to an existing Human Interface resource, it may contain information about the 
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path-name of the resource; the type of resource (e.g. picture, window, or terminal); and the name of the file 

(for pictures only) from which to load the picture. This request can be sent to a Human Interface manager or 

a console manager; alternatively the same message with message I.D. "init" specifying a file can be sent 

directly to a privately owned picture manager. 
5 DELETE is used to remove an existing Human Interface resource from the system, and it may contain 

information specifying a connection to the resource; the type of resource; and whether, for a window, the 

corresponding picture is to be deleted at the same time. 

CLOSE is used to break connection to a Human Interface resource, and it may contain information 

specifying a connection to the resource; and the type of resource. 
w WHO? is used to get the status of a service or resource, and it may contain a user identification string. 

QUERY is used to get the status of a service or resource, and it may contain information about the 

resource type; the name of the service or resource; a connector to a resource; and information concerning 

various options. 

The following are the various Connection Responses and the types of information which may be 
'5 associated with each: 

CONNECT provides a connection to a Human Interface resource, and it contains information concerning 
the originator (i.e. the Human Interface or the console); the resource type; the original request message 
identifier; the name of the resource; and a connector to the resource. 

USER contains the names of zero or more currently signed-on users and their locations, and it contains 
20 a connector to a console manager followed by the name of the user signed on at that console. 



CONSOLE REQUESTS 

25 The main purpose of the console is to coordinate the activities of the windows, pictures, and dialog 
associated with it. Any of the CREATE. OPEN, DELETE, and CLOSE connection requests listed above, 
except those relating to the consoles, can be sent directly to a known console manager, rather than to the 
Human Interface manager {which always searches for the console by name). Subsequently, some char- 
acteristics of a window, such as its size, can be changed dynamically through the console manager. The 
30 current "user" of the console can be changed. And the console can be queried for its current status (or that 
of any of its resources). 

The following are the various Console Requests and the types of information which may be associated 
with each: 

USER is used to change the currently signed-on user, and it contains a user identification string. 
CHANGE is used to change the size and other conditions of a window, and it may contain information 
about a connector to a window or a terminal; new height and width (in virtual pixels); increment to height 
and width; row and column position; various options; a connector to a new owner process; and whether the 
window should be the current active window on the screen. 

CURSOR is used to move the screen cursor, and it contains position information as to row and column. 
QUERY is used to get the current status of the console or one of its resources, and it contains 
information in the form of a connector to the resource; and various query options (e.g. list all screens, all 
pictures, or all windows). 

BAND starts/stops the rubber-banding function and dragging function, and it contains information about 
the position of a point in the picture from which to start the operation; the end point of the figure which is to 
be dragged; the type of operation (e.g. line, rectangle, circle, or ellipse); the color; and the- type of line (e.g. 
solid). In rubber-banding the drawn figure changes in size as the cursor is moved, in dragging the figure 
moves with the cursor. 

The following are the various Console Responses and the types of information which may be 
associated with each: 

STATUS describes the current state of a console, and it may contain information about a connector to 
the console; the originator; the name of the console: current cursor position; current metaphor size; scale of 
virtual pixels per centimeter, vertically and horizontally; number of colors supported; current user i.d. string; 
screen size and name; window connector and name: and picture connector, screen name, and window 
name. 
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PICTURE-DRAWING 

The picture is the fundamental building block in the Human Interface. It consists of a list of zero or 
more "picture elements", each or which is a device-independent abstraction of a displayable object (line, 
s text, etc.). Each currently active picture is stored and maintained by a separate picture manager. "Drawing" 
a picture consists of sending picture manipulation messages to the picture manager. 

A picture manager must first be initialized by a CREATE or OPEN request (or INIT. if the picture was 
created privately). CREATE sets the picture to empty, gives it a name, and defines the background. The 
OPEN request reads a predefined picture from a file and gives it a name. Either must be sent first before 
jo anything else is done. A subsequent OPEN reloads the picture from the file. 

The basic request is to WRITE one or more elements. WRITE adds new elements to the end of the 
current list, thus reflecting the order. Whenever parts of the picture are copied or displayed, this order is 
preserved. Once drawn, one or more elements can be moved, erased, copied, or replaced. All or part of the 
picture can be saved to a given file. In addition, there are requests to quickly change a particular attribute of 
is one or more elements (e.g. select then). Finally, the DELETE request (to the console manager; QUIT, if 
direct to the picture resource) terminates the picture manager, without saving the picture. 

A picture can be shared among several processes ("applications") by setting the "appl" field in the 
picture elements. Each application process can treat the picture as if it contains only its own elements. All 
requests made by each process will only affect elements which contain a matching "appl" field. Participate 
20 ing processes must be identified to the picture manager via an "appl" request. 

The following are the various Picture-Drawing Requests and the types of information which may be 
associated with each: 

WRITE is used to add new elements to a picture, and it may contain information providing a list of 
picture elements; the data type; and an indication to add the new elements after the first element found in a 
25 given range (instead of the foreground, at the end of the list). 

READ is used to copy elements from a picture, and it may contain information regarding the connection 
to which to send the elements; an indication to copy background elements; and a range of elements to be 
copied. 

MOVE is used to move elements to another location, and it may contain information indicating a point in 
30 the picture to which the elements are to be moved; row and column offsets; to picture foreground: to picture 
background; fixed size increments; and a range of elements to be moved. 

REPLACE is used to replace existing elements with new ones, and it may contain information providing 
a list of picture elements; and a range of elements to be replaced. 

ERASE is used to remove elements from a picture, and it may contain information on the range of 
35 elements to be erased. 

QUIT is used to erase all elements and terminate, and it has no particular parameters (valid only if the 
picture is private). 

MARK is used to set a "marked" attribute (if text, to display a mark symbol), and it may contain 
information specifying the element to be marked; and the offset of the character after which to display the 
40 mark symbol. 

SELECT is used to select an element and mark it, and it may contain information specifying the 
element(s) to be selected; the offset of the character after which to display the mark symbol; the number of 
characters to select; and a deselect option. 

SAVE is used to copy all or part of a picture to a file, and it may contain information specifying the 
45 name of the file; and a subset of a picture. 

QUERY is used to get the current status, and it has no particular parameters. 

BKGD is used to change a picture's background color, and it may contain information specifying the 
color. 

APPL is used to register a picture as an "application; a may contain information specifying a name of 
so the application; a connection to the application process; and a point of origin inside the picture. 

NUMBER is used to get ordinal numbers and identifiers of specific elements, and it may contain 
information specifying the element(s). 

HIT is used to find an element at or closest to a given position, and it may contain a position location in 
a picture; and how far away from the position the element can be. 
55 [,] is used to start'end a batch, and a first symbol causes all updates to be postponed until a second 
symbol is received (batches may be nested up to 10 deep). 

HIGHLIGHT, INVERT] BLINK, HIDE are used to change a specific element attribute, and they may 
contain information indicating whether the attribute is set or cleared; and a range of elements to be 
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changed. 

CHANGE is used to change one or more of the element fields, and it may contain information 
specifying the color of the element; the background color; the fill color; and fill pattern; and a range of 
elements to be changed. 

5 EDIT is used to modify a text element's string, and it may contain information indicating to edit at the 
current mark and then move the mark; specifying the currently selected substring is to be edited; an offset 
into the text at which to insert and/or from which to start shifting; to shift the text by the given number of 
characters to- from the given position; tab spacing; a replacement substring; to blank to the end of the 
element; and a range of elements to be edited. 

to The following are the various Picture-Drawing responses and the types of information which may be 
associated with each: 

STATUS describes the current status of the picture, and it may contain information specifying a 
connector to the picture; an original message identifier, if applicable; the name of the picture; the name of 
the file last read or written; height and width; lowest and highest row/column in the picture; the number of 
/5 elements; and the number of currently active viewports. 

WRITE contains elements copied from a picture, and it may contain information specifying a connector 
to the picture; a list of picture elements; and the data type. 

NUMBER contains element numbers and identifiers, and it may contain information specifying a list of 
numbers; and a list of element identifiers. 



PICTURE ELEMENTS 



Picture elements are defined by a collection of data structures, comprising one for a common "header", 
25 some optional structures, and one for each of the possible element types. The position of an element is 
always given as a set absolute coordinates relative to [0,0] in the picture. This defines the upper left corner 
of the "box" which encloses each element. Points specified within an element (e.g. to define points on a 
line) are always given as coordinates relative to this position. In a "macro" the starting position of each 
individual element is considered to be relative to the absolute starting position of the macro element itself, 
30 i.e. they're nested. 

FIG. 7 shows the general structure of a complete picture element. The "value" part depends upon the 
element type. The "appl" and "tag" fields are optional, depending upon indicators set in "attr". 
The following is a description of the various fields in a picture element: 

35 Length = length of the entire picture in bytes 

Type =one of the following: text. line, rectangle, ellipse, circle, symbol, array, discrete, macro, null, 
metaelement 

Attr = one of the following: selectable, selected, rectilinear, inverted foreground/background, blink, 
tagged, application mnemonic, hidden, editable, movable, copyable, erasable, transformed, highlighted. 
40 mapped/not mapped, marked, copy 

Pos = Row/col coordinates of upper left corner of the element's box 

Box = Height/width of an imaginary box which completely and exactly encloses the element 
Color = color of the element, consisting of 3 sub-fields: hue, saturation, and value 
Bkgrnd = background color of the element 
45 Fill = the color of the interior of a closed figure 
Pattern = one of 10 "fill" patterns 

Appl = a mnemonic referencing a particular application (e.g. forms manager, word-processor, report 
generator, etc.); allows multiple processes to share a single picture. 

Tag = a variable-length, null-terminated string, supplied by the user: it can be used by applications to 
so identify particular elements or classes of elements, or to store additional attributes 



The attributes relating to the "type" field if designated "text" are as follows: 

Options = wordwrap, bold, underline, italic, border, left-justify, right-justify, centered, top of box, bottom 
55 of box. middle of box, indent, tabs, adjust box size, character size, character-line spacing, and typeface 
Select = indicates a currently selected substring by offset from beginning of string, and length 
String = any number of bytes containing ASCII codes, followed by a single null byte; the text will be 
constrained to fit within the element's "box", automatically breaking to a new row when it reaches the right 
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boundary of the area 

Indent = two numbers specifying the indentation of the first and subsequent rows of text within the 
element's "box" 

Tabs = list of [type, position], where "position" is the number of characters from the left edge of the 
5 element's box, and "type" is either Left, Right, or Decimal 

Grow = maximum number of characters (horizontally) and lines (vertically) by which the element's box 

may be extended by typed input; limits growth right and downward, respectively 

Size = height of the characters' extend and relative width 

Space = spacing between lines of text and between characters 
io Face = name of a particular typeface 

- The attributes relating to the "type" field if designated "line" are as follows: 

Style = various options such as solid, dashed, dotted, double, dashed-dotted, dash-dot-dot, patterned. 
is etc. 

Pattern = a pattern number 
Thick = width of the line in pixels 

Points = two or more pairs of coordinates (i.e. points) relative to the upper left corner of the box 

defined in the header 
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The attributes relating to the "type" field if designated "rectangle" are as follows: 



Style = same as for "line" above, plus solid with a shadow 
Pattern = same as for "line" 

25 Thick = same as for "line" 

Round = radius of a quarter-circle arc which will be drawn at each corner of the rectangle 

The attributes relating to the "type" field if designated "ellipse" are as follows: 

30 Style = solid, patterned, or double 
Pattern = same as for "line" 
Thick = same as for "line" 

Arc = optional start-and end-angles of an elliptical arc 

as The attributes relating to the "type" field if designated "circle" are as follows: 

Style = same as for "ellipse" 

Pattern = same as for "line" 

Thick = same as for "line" 

40 Center = a point specifying the center of the circle, relative to the upper left corner of the element's 
box 

Radius = length of the radius of the circle 

Arc = optional start-and end-angles of a circular arc 

45 A "symbol" is a rectangular space containing pixels which are visible (drawn) or invisible (not drawn). It 
is represented by a two-dimensional array, or "bit-map" of 1's and O's with its origin in the upper left 
corner. 

The attributes relating to the "type" field if designated "symbol" are as follows: 

so Bitmap = a two-dimensional array (in row and column order) containing single bits which are either "1 " 
(draw the pixel in the foreground color) or "0" (draw the pixel in the background color); the origin of the 
array corresponds to the starting location of the element 

Alt = A text starting which can be displayed on non-bit-mapped devices, in place of the symbol 

55 An array element is a rectangular space containing pixels which are drawn in specific colors, similar to 
a symbol element. It is represented as a two-dimensional array, or "bit-map", of color numbers, with its 
origin in the upper left corner. The element's "fill" and "pattern" are ignored. 
The attributes relating to the "type" field if designated "array" are as follows: 
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Bitmap = a two-dimensional array (in row and column order) of color numbers; each number either 
defines a color in which a pixel is to be drawn, or is zero (in which the pixel is drawn in the background 
color); the origin of the array corresponds to the starting location of the element 
5 Ait = an alternate text string which can be displayed on non-bit-mapped devices in place of the array 

A discrete element is used to plot distinct points on the screen, optionally with lines joining them. Each 
point is specified by its coordinates relative to the element's "box". An explicit element (usually a single- 
character text element or a symbol element) may be given as the mark to be drawn at each point. If not, an 
to asterisk is used. The resulting figure cannot be filled. 

The attributes relating to the "type" field if designated "discrete" are as follows; 

Mark = a picture element which defines the "mark" to be drawn at each point; if not applicable, a null- 
length element (i.e., a single integer containing the value zero) must be given for this field 
75 Style 
Pat 

Thick = type, pattern, and thickness of the line (see "line" element above) 

Join = "Y" or "N" (or null, which is equivalent to "N"); if "Y", lines will be drawn to connect the marks 
Points = two or more pairs of coordinates; each point is relative to the upper left corner of the "box" 
20 defined in the header 

A "macro" element is a composite, made up of the preceding primitive element types ("text", etc.) 
and/or other macro elements. It can sometimes be thought of as "bracketing" other elements. The 
coordinates of the contained elements are relative to the absolute coordinates of the macro element. The 
25 "length" field of the macro element includes its own header and the "macro" field, plus the sum of the 
lengths of all of the contained elements. The "text" macro is useful for mixing different fonts and styles in 
single "unit" (word, etc.) of text. 

The attributes relating to the "type" field if designated "macro" are as follows: 

30 Macro = describes the contents of the macro element; may be one of following: 
"N" - normal (contained elements are complete) 

"Y" - list: same as "N", but only one sub-element at a time can be displayed; the others will be 
marked "hidden", and only the displayed element will be sent in response to requests ("copy, 
etc.); the "highlight" request will cycle through the sub-elements in order 
35 " T " * text: same as "N", but the "macro" field is immediately followed by a text "options" field, 

and a text "select" field; the macro "list" field may be followed by further text parameters (as 
specified in the options field) 

List = any number of picture elements (referred to as sub-elements), formatted as described above; 

terminated by a null word 

40 

A "meta-element" is a pseudo-element generated by the picture manager and which describes the 
picture itself, whenever the picture is "saved" to a file. Subsequently, meta-elements read from a file are 
used to set up parameters pertinent to the picture, such as its size and background color. Meta-elements 
never appear in "write" messages issued by the picture manager (e.g. in response to a "read" request, or 
45 as an update to a window manager). 

The format of the meta-element includes a length field, a type field, a meta-type field, and a value. The 
16-bit length field always specifies a length of 36. The type field is like that for normal picture elements. 
The meta-element field contains one of the following types: 

so Name = the value consists of a string which names the picture 

Size = the maximum row and column, and the maximum element number and size 
8ackgnd = the picture's background color 
Highlt = the picture's highlighting 

55 The format of the value field depends upon the meta-type. 
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WINDOWING 

A window maps a particular subset (often called a "view") of a given picture onto a particular screen. 
Each window on a screen is a single resource which handles the "pane" in which the picture is displayed 
5 and up to four "frame bars". 

With reference to FIG. 8, a frame bar is used to show ancillary information such as a title. Frame bars • 
can be interactive, displaying the names of "pull-down" menus which, when selected, display a list of 
options or actions pertinent to the window. A palette bar is like a permanently open menu, with all choices 
constantly visible. 

w Scroll bars indicate the relative position of the window's view in the picture and also allow scrolling by 
means of selectable "scroll buttons", A "resize" box can be selected to expand or shrink the size of the 
window on the screen while a "close" box can be selected to get rid of the window. Selecting a "blow-up" 
box expands the window to full screen size; selecting it again reduces it to its original dimensions. 
A corner box is available for displaying additional user information, if desired. 

75 The window shown in FIG. 8 comprises a single pane, four frame bars, and a corner box. The 
rectangular element within each scroll bar indicates the relative position of the window in the picture to 
which it is mapped (i.e. about a third of the way down and a little to the right). 

Performing an action( such as a "select") in any portion of the window will optionally send a "click" 
message to the owner of the window. For example, selecting an element inside the pane will send "click" 

20 with "action" = "select" and "area" = "inside", as well as the coordinates of the cursor (relative to the top 
left corner of the picture) and a copy of the element at that position. 

Selecting the name of a menu, which may appear in any frame bar, causes the menu to pop-tfp. It is 
the response to the menu that is sent in the "click" message, not the selection of the menu bar item. Pop- 
up menus (activated by menu keys on the keyboard) and function keys can also be associated with a 

25 particular window. 

All windows are created by sending a "create" request to a Console Manager. As described above, 
"create" is the most complex of the windowing messages, containing numerous options which specify the 
size and location of the window, which frame bars to display, what to de when certain actions are performed 
in the window, and so on. 

30 The process which sent the request is known as the "owner" of the window, although this can be 

changed dynamically. The most recently opened window usually becomes the current "active" window, 

although this may be overridden or changed. 

A subsequent "map" request is necessary to tell the window which picture to display (if not specified in 

the "create" request). "Map" can be re-issued any number of times. 
35 Other requests define pop-up menus and soft-keys or change the contents of specific frame bars. A 

window is always opened on top of any other window(s) it overlaps. Depending upon the background 

specified for the relevant picture, underlying windows may or may not be visible. 

The "delete" request unmaps the window and causes the window manager to exit. The owner of the 

window (if different from the sender of "delete") is sent a "status" message as a result. 
40 The following are the various Windowing Requests and the types of information which may be 

associated with each: 

MAP is used to map or re-map a picture to the window, and it may contain information specifying a 
connection to the desired picture; and the coordinates in the picture of the upper left corner of the 
"viewport", which will become [0.0] in the window's coordinate system. 
4$ UNMAP is used to disconnect a window from its picture, and it contains no parameters, 

QUERY is used to get a window's status, and it contains no parameters. 

[J is used to started a "batch", and the presence of a first symbol causes all updates to be postponed 
until a second symbol is received (batches may be nested up to 10 deep). 

MENU defines a menu which will "pop-up" when a menu key is pressed, and it may contain information 
so specifying which menu key will activate the menu; the name of the menu in the Human Interface library (if 
omitted, "list" must be given); and a name which is returned in the "click" message. 

KEYS defines "pseudo-function" keys for the window, and it may contain information specifying the 
name of a menu in the Human Interface library; a list of key-names; and a name to be returned in the 
"click" message. 

55 ADD, COPY, ERASE, REPLACE control elements in a frame bar, and they may contain information 
specifying the type of bar (e.g. title, palette, general, etc.); a list of picture elements for "add" and "replace" 
(omitted for "copy" and "erase"); and a tag identifying a particular element (not applicable to "add"). 

HIGHLIGHT, INVERT, HIDE, BLINK change attributes in a frame bar element, and they may contain 
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information specifying a set/clear attribute; the type of bar; and a tag identifying a particular element in the 
bar. 

The following are the various Windowing responses and the types of information which may be 
associated with each: 

STATUS describes the current status of the window, and it may contain information specifying a 
connector to the window; specifying the originator (i.e. "window"); an original message identifier, if ap- 
plicable; the subsystem; the mane of the window; a connector to the window's console manager; the 
position of the window on the screen: the pane size and location: a connector to the picture currently 
mapped to the window; and the size and position of the view. 

BAR represents a request to a "copy" request, and it may contain information specifying the type of 
bar (e.g. title, palette, general, corner box, etc.); and a list of picture elements, 

CLICK describes a user-initiated event on or inside the window, and it may contain information 
specifying what event (e.g. inside a pane, frame bar, corner box, pop-up menu, function key, etc.); a 
connector to the window manager; a connector to the window's Console Manager; the name of the window; 
a menu or function-key name; a connector to the associated picture manager; a label from a menu or 
palette bar item or from a function key; the position of the cursor where the action occurred; the action 
performed by the user; a copy of the elements at the particular position; the first element's number; the first 
element's identifier: a copy of the character typed or a boundary indicator or the completion character; and 
other currently selected elements from ail other windows, if any. 



HI - DETAILED DESCRIPTION 

USER-ADJUSTABLE WINDOW 

Figure 9 illustrates the relationship between pictures, windows, the console manager (which creates and 
destroys the objects), and a virtual output manager (which performs output to physical devices). In response 
to one or more application programs 225, the console may also create at least one window for viewing a 
portion of each picture. The virtual output manager 235 translates the virtual output corresponding to each 
30 window into a form suitable for display on a 'Veal" output device such as a video display terminal. 

One or more of windows 231-233 can be displayed simultaneously on output device 236. While 
windows 231-233 are shown to display portions of separate pictures, they could just as well display different 
portions of single picture. 

FIG. 10 shows a flowchart illustrating how an application program interacts with the console manager 
as process to create and/or destroy windows and pictures. In response to application requests 240 the console 
manager 241 can proceed to an appropriate program module 242 to create a picture 244 or a window 243, 
or to module 245 to destroy a window 246 or a picture 247. 

If the console manager is requested to create a new window 234, it first starts a new window process. 
Then it initializes the window by drawing the frame, etc. Then it defines the initial view of the given picture. 
40 If the console manager is requested to create a new picture 244, it starts a new picture process. 
If the console manager is requested to delete a window 246, it closes the window. 
If the console manager is requested to delete a picture 247, it tells the picture to quit 
FIG. 1 1 illustrates an operation to update a picture and see the results in a window of selected size, in 
accordance with a preferred embodiment of the present invention. The operation performed in FIG. 14 
45 corresponds to that indicated by line segment 201 in FIG. 12. 

In response to a request from an application 249, the picture manager 250 may perform any of the 
indicated update actions. For example, the picture manager 250 may change the view of the picture by 
allocating a descriptor and accordingly filling in the location and size of the view. 

Or the picture manager 250 may draw, replace, erase, etc. picture elements appropriately as requested, 
so It repeats the requested operation for each view. 



PICTURE - LIVE DATA PROM MULTIPLE APPLICATIONS 

FIG. 12 illustrates how a single picture can share multiple application software programs. A picture 265 
can include any number of independent applications, such as spread-sheet 260, graphic package 262, 
word-processing 264, data base management 268. and process control 266. appointment calendar (not 
shown), etc. Each application attaches meaning to the particular organization of picture elements under its 
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control, by interpreting them as a spreadsheet, graph, a page of formatted document, etc. 

FIG. 13 illustrates how the picture manager multiplexes several applications to a single picture. Picture 
manager 276 keeps track of the picture elements belonging to each application 271-275. Any requests it » 
receives to access or modify the picture are checked against the list of constituent applications. Picture 
s elements not belonging to the application making the current request are simply skipped. 

Picture manager 276 can perform draw, copy, replace, erase, and or other operations upon the 
appropriate picture elements of applications 271-275. 

The Human Interface allows multiple applications to share a single picture, so that spreadsheets, 
graphs, and text (for example) can be combined to suit a particular user. For example, FIG. 14 illustrates 
to the live integration of two applications on a single screen. Portion 291 shown on the screen represents text * 
from a text editing or word-processing application. Portion 291 is fully editable by the user. 

Portion 292 represents a portion of a spread-sheet application, and it too is fully modifiable by the user. 
The modification of the contents of any cell of the spread-sheet will reflect appropriate changes to the - 
portion 292 being displayed on the screen illustrated in FIG 14. 
is Regarding the picture comprising the word-processing and spread-sheet applications shown in FIG. 14, 
neither of the applications is aware of the existence of the other, nor is it aware of, or affected by, the fact 
that the picture is being shared. 

Each application operates as if it were the sole user of the picture. The net effect (on an output device, 
such as a VDT screen) is a single, cohesive visual image, updated dynamically by an or all of the relevant 
so applications, totally independently of each other. 

INPUT OUTPUT DEVICE INDEPENDENCE 

25 In the present invention all system interaction with the outside world is either through "virtual input" or 
"virtual output" devices. The system can accept any form of input or output device. The Human Interface is 
constructed using a well-defined set of "virtual devices". All Human Interface functions (windowing, picture- 
drawing, dialog management, etc.) use this set of devices exclusively. 

These virtual input devices take the form of "keys" (typed textual input"; "position (screen coordinates); 
30 "actions" (Human Interface functions such as "open window", etc.) "functions" (user-defined actions); and 
"means" (pop-up lists of choices). 

Virtual output devices produce device-independent output: text, lines, rectangles, polygons, circles, 
ellipses, discrete points, bit-mapped symbols, and bit-mapped arrays. 

FIG. 15 shows how the console manager operates upon virtual input to generate virtual output. The 
35 lowest layer of HI software converts input from any "real" physical devices to the generic, virtual form, and 
it converts Human Interface output (in virtual form) to physical output. 

Figure 15 shows the central process of the Human Interface, the console manager 300, dealing with 
virtual input and producing virtual output. Virtual input passes through the virtual input manager 301 is 
processed directly by the console manager 300, while output is passed through two intermediate processes 
40 - (1) a picture manager 302, which manipulates device-independent graphical images, and (2) a window,, 
manager 304, which presents a subset (called a "view") of the overall picture to the virtual output manager 
306. 

Any number of physical devices can be connected to the Human Interface and can be removed or 
replaced dynamically, without disturbing the current state of the Human Interface or of any applications 
45 using the Human interface. In other words, the Human Interface is independent of particular I/O devices, ' 

and the idiosyncracies of the devices are hidden from the Human Interface. * 

FIG. 16 represents a flowchart showing how virtual input is handled by the console manager. The virtual 
input may take any of several forms, such as a keystroke, cursor position, action, function key. menu, etc. 

For example, regarding the operations beneath block 311, if the virtual input to the console manager is * 
50 keystroke, then the console manager checks to see whether the cursor is inside a window. If so, it checks 
to see whether it originated from a virtual terminal, and if not it checks to see whether an edit operation is 
taking place. If not, it updates the picture. 

Regarding the operations beneath block 312, if the virtual input represents a cursor position, then the 
console manager checks to see whether the auto-highlight option has been enabled. If yes, it checks to see 
55 whether the cursor is on an element. If so it highlights that element. 

Regarding the operations beneath block 313, the console manager uses any of the indicated actions to 
update a picture, update a window, or initiated dialog, as appropriate. 

Regarding the operations beneath block 314, if the virtual' input is from a function key, the console 
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manager notifies the diaiog manager. 

Regarding the operations beneath block 315. if the virtual input represents a menu choice, the console 
manager checks to see whether the cursor is in a window. If not, it determines that it is on a user metaphor; 
if so. it requests a menu from the window. If the menu is defined, it notifies the owner of the window (or 

5 metaphor), activates a pop-up menu, gets a response, and sends the response to the window owner. 

FIG. 17 represents a flowchart showing how virtual input is handled by the picture manager. The picture 
manager 320 accepts virtual output from the console manager and then, depending upon the type of 
operation, performs the requested function. For example, if the operation is a replace operation, the picture 
manager 320 replaces the old output with the new and sends the change(s) to the window manager. The 

io window manager sends the change to the output manager, which in turn sends it to the real device. 



SCREEN - LIVE DATA IN MULTIPLE WINDOWS 

ts FIG. 18 illustrates how the console manager 340 enables multiple application software programs 330- 
334 to be represented by multiple pictures 314-343. and how multiple windows 361-363 and 367 may 
provide different views of one picture. 

Console manager 340, in response to requests, can create or open application processes, such as 
process control module 330, spread-sheet module 331. graphics package 332, word-processing software 

20 333. or data base management 334, on any or all of pictures 341-343. Window 361 may view a portion of 
picture 341; window 362 views a portion of picture 342; and windows 363 and 367 may view different 
portions of picture 343. The virtual output of window managers 361-363 and 367 is processed by the virtual 
output manager 365, which also transforms it into a form suitable to be displayed by a real output device, 
such as a video display terminal 366. 

25 FIG. 19 illustrates how several windows may be displayed simultaneously on a typical screen. The 
Human Interface allows portions of multiple applications to be displayed via separate windows. For example, 
FIG. 19 shows the simultaneous display of a live text portion 371 from a word-processing application, a live 
numerical portion 370 from a spread-sheet and a live graphic portion 372 from a graphics program. The 
information in each window 370-372 is "live", in that it may change according to the results of on-going 

30 processing. 

The user may add or modify information in windows 370-372 at any time, and any changes in the 
information displayed will take effect in the appropriate window(s) as it is processed. For example, a change 
to one application display in one window could result in changes to information displayed in several 
windows. 

35 

Description of Source Code Listings 
User-Adjustable Window 

40 

Program Listings A and 8 contain a X" language implementation of the concepts relating to adjusting 
the size of a display window as described hereinabove. The following chart indicates where the relevant 
portions of the listings may be found. 
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Function 



Line Numbers in 
Program Listing A 



10 



J5 



Main-line: initialization; accept requests 190-222 

Determine type of request 329-369 

Create: 418-454 

Create a window 1298-1600 

Create a picture 440-447 

Destroy (delete) 456-484 
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Function 



Line Numbers in 
Program Listing B 
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30 



Main-line: initialization; start processing 125-141 

Accept requests; check for changes 161-203 

Determine type of request 239-310 

View: 1205-1249 

Draw: 410-457 

Replace : 537-585 

Erase: 587-609 



35 Picture - Live Data From Multiple Applications 

Program Listing 8 contains a "C" language implementation of the concepts relating to accepting 
requests to modify elements of applications simultaneously resident in a single picture as described 
hereinabove. The following chart indicates where the relevant portions of the listing may be found. 
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Function 



Main-line: initialization; start processing 

Accept requests; check for changes 

Determine type of request 

Register application 

Draw, copy, etc. 

Check if application registered 

Check if element belongs to application 



Line Numbers in 
Program Listing B 

124-141 
161-213 
239-310 
843-864 
312-841 
179, 180, 205-217 
1653-1659 
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Input; Output Device independence 

Program Listings A and 8 contain a "C" language implementation of the above-described concepts 
relating to input/output device independence. The following chart indicates where the relevant portions of 
5 the listing may be found. 



jo 



Function 



Lines Numbers in 
Program Listing A 



15 



20 



Main-line; initialization; accept input 

Determine type of input 

Virtual key 

Virtual position 

Virtual action 

Virtual function 

Virtual menu 



190-222 
486-521 
523-631 
633-661 
663-702, 763-1200 
704-723 
725-761 
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Function 



Lines Numbers in 
Program Listing B 



30 



35 



40 



Main-line; initialization; start processing 125-141 

Accept requests (virtual output) ; check for 161-203 
changes 

Determine type of request 239-310 

Draw 410-457 

Copy 611-632 

Replace 537-585 

Erase 587-609 

Move 634-678 

Send changes 1265-1352 
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50 



Screen - Live Data in Multiple Windows 

Program Listing contains a "C" language implementation of the concepts relating to the simultaneous 
display of "live" windows from multiple applications on a single screen as described hereinabove. The 
following chart indicates where the relevant portions of the listing may be found. 



55 
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Function Line Numbers in 

Program Listing B 

5 

Main-line: initialization; start processing 124-141 

Accept requests; check for changes 161-213 

jo Determine type of request 239-310 

Register application 843-864 

Draw, copy, etc. 312-841 

Qieck if application registered 179, 180, 205-217 

Check if element belongs to application 1653-1659 

It will be apparent to those skilled in the art that the herein disclosed invention may be modified in 
numerous ways and may assume many embodiments other than the preferred form specifically set out and 
described above. For example, its utility is not limited to a data processing system or any other specific 
type of data processing system. 

Accordingly, it is intended by the appended claims to cover al! modifications of the invention which, fall 
within the true spirit and scope of the invention. 
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Claims 

1. A human interface in a data processing system, said interface comprising: 

means for representing information in at least one abstract device-independent picture (343, FIG. 18); 

means (330-334) for generating a first message, said first message comprising size information; and 
a console manager process (340) responsive to said first message for creating a window (363) onto 
said one picture, the size of said window being determined by said size information contained in said first 
message. 

2. The human interface is recited in claim 1 and further comprising: 

means (330-334) for generating a second message, said second message comprising size information; 

and 

said console manager process being responsive to said second message for creating a second window 
(367) onto said picture, the size of said second window being determined by said size information contained 
in said second message, the sizes of said window and said second window being independent of one 
another. 

3. The human interface as recited in claim 1 and further comprising: 

means (330-334) for generating a second message, said console manager process being responsive to 
said second message for creating an additional picture (342). 

4. The human interface as recited in claim 3 and further comprising: 

means (330-334) for generating a third message, said third message comprising information for 
modifying said one picture and said additional picture; and 

a picture manager process (276, FIG. 13) responsive to said third message for modifying both said one 
picture and said additional picture simultaneously in accordance with said information. 

5. A human interface in a data processing system, said interface comprising: 

means for representing information in at least one abstract, device-independent picture (221, FIG. 9); 

and 

means permitting said picture to be shared by a plurality of independent applications (301, 303, FIG. 

6). 

6. The human interface as recited in claim 5, and further comprising a plurality of abstract, device- 
independent pictures (341-343. FIG. 18); and 

means permitting each of said pictures to be shared by a plurality of independent applications. 
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7. The human interface as recited in claim 5. wherein said picture comprises user interface information, 
said human interface further comprising: 

means for simultaneously displaying images from at least one of said applications and from said user 
interface information (FIGS. 14. 19). 

8. The human interface as recited in claim 7, wherein said user interface information includes 
information from the group comprising menu information, icon information, help information, and prompt 
information, and wherein said at least one application is from the group comprising a text-editing 
application, a spread-sheet application, a graphics application, a database application, and a process control 
application. 

9. A virtual input interface in a data processing system, said interface comprising: 
means (301, FIG. 15) for accepting input from at ieast one physical device; 
means for converting said physical device input into virtual input; and 

means (300) responsive to said virtual input for performing processing operations upon said virtual 

input. 

10. The virtual input interface as recited in claim 9. wherein said at least one physical device can be 
removed from said system without affecting the operation of the remainder of said system. 

1 1 . The virtual input interface as recited in cfaim 9. wherein at least one additional physical device can 
be added to said system without affecting the operation of the remainder of said system. 

12. A virtual output interface in a data processing system, said interface comprising: 

means (306, FIG. 15) for accepting virtual output generated by system processing operations; and 
means for converting said virtual output into at least one physical output suitable for use by at least one 
physical device. 

13. The virtual output interface as recited in claim 12. wherein said at least one physical device can be 
removed from said system without affecting the operation of the remainder of said system. 

14. The virtual output interface as recited in claim 12, wherein at least one additional physical device 
can be added to said system without affecting the operation of the remainder of said system. 

15. A human interface in a data processing system, said interface comprising: 

means (343, FIG. 18) for representing information in at least one abstract, device-independent picture: 
means (301, 303, FIG. 6) permitting said picture to be shared by a plurality of independent 
applications; and 

means permitting live information from said picture to be displayed in more than one window 
simultaneously. 

16. The human interface as recited in claim 15, and further comprising a plurality of abstract, device- 
independent pictures (341-343, FIG. 18); and 

means permitting each of said pictures to be shared by a plurality of independent applications. 

17. The human interface as recited in claim 15, wherein said picture comprises user interface 
information, said human interface further comprising: 

means (370-372, FIG. 19) for simultaneously displaying images from at least one of said applications 
and from said user interface information. 

18. The human interface as recited in claim 17, wherein said user interface information includes 
information from the group comprising menu information, icon information, help information, and prompt 
information, and wherein said at least one application is from the group comprising a text-editing 
application, a spread-sheet application, a graphics application, a database application, and a process control 
application. 
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© Computer human interface. 



© In a computer human interface an adjustable 
"window" (177, FIG 4) enables the user to view a 
portion of an abstract, device-independent "picture" 
description of information. More than one window 
can be opened at a time. Each window can be sized 
independently of another, regardless of the applica- 
tions running on them. The human interface creates 
a separate "object" (represented by a process) for 
each active picture and for each active window. The 
pictures are completely independent of each other. 
Multiple pictures (170, 174) can be updated simulta- 
neously, and windows can be moved around on the 
< screen and their sizes changed without the involve- 
l^ment of other windows and/or pictures. Images, in- 
qq eluding windows, representing portions of any or all 
Oof the applications can be displayed and updated on 
^the output device simultaneously and independently 
^sof one another. All human interface with the operat- 
CMing system is performed through virtual input/output 
0 devices (186, 187, FIG. 5). and the system can 
^accept any form of real input or output devices, 
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